Computer databases have become extremely sophisticated, e.g., the computing demands placed on database systems have increased at a rapid pace. Database systems are typically configured to separate the process of storing data from accessing, manipulating or using data stored in the database. More specifically, databases use a model where data is first stored, then indexed, and then queried. However, this model cannot meet the performance requirements of some real-time applications. For example, the rate at which a database system can receive and store incoming data can limit how much data can be processed or otherwise evaluated, which, in turn, limits the utility of database applications configured to process large amounts of data in real-time.
To address this issue, stream based computing and stream based database computing is emerging as a developing technology for database systems, where products are available and allow users to create applications that process and query streaming data before it reaches a database file. With this emerging technology, users can specify processing logic to apply to inbound data records while they are “in flight,” with the results available in milliseconds. Constructing an application using this type of processing has opened up a new programming paradigm that will allow for a broad variety of innovative applications, systems and processes to be developed as well as present new challenges for application programmers and database developers.
One or more data streams may provide a continuous flow of data tuples from source processing elements to destination processing elements. In some cases, non-pertinent data tuples are transmitted between the processing elements, resulting in unnecessary consumption of processing power and bandwidth.